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Abstract 

To inform clinical guidelines and patient care we need high quality evidence on the relative benefits and harms of 
intervention. Patient reported outcome (PRO) data from clinical trials can "empower patients to make decisions 
based on their values" and "level the playing field between physician and patient". While clinicians have a good 
understanding of the concept of health-related quality of life and other PROs, evidence suggests that many do not 
feel comfortable in using the data from trials to inform discussions with patients and clinical practice. This may in 
part reflect concerns over the integrity of the data and difficulties in interpreting the results arising from poor 
reporting. 

The new CONSORT PRO extension aims to improve the reporting of PROs in trials to facilitate the use of results to 
inform clinical practice and health policy. While the CONSORT PRO extension is an important first step in the 
process, we need broader engagement with the guidance to facilitate optimal reporting and maximize use of PRO 
data in a clinical setting. Endorsement by journal editors, authors and peer reviewers are crucial steps. Improved 
design, implementation and transparent reporting of PROs in clinical trials are necessary to provide high quality 
evidence to inform evidence synthesis and clinical practice guidelines. 
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Background 

Randomized controlled clinical trials (RCTs) can provide 
high-quality data regarding the impact of study interven- 
tions on patient outcomes, and remain the 'gold stand- 
ard' of evidence regarding both the benefits and harms 
associated with an intervention. Over the last twenty 
years, the number of clinical trials that assess patient re- 
ported outcomes (PROs) has substantially increased [1]. 
PROs can be defined as a "measurement of any aspect of 
a patient's health status that comes directly from the pa- 
tient (i.e. without the interpretation of the patient's re- 
sponses by a physician or anyone else" [2] and include 
health-related quality of life (HRQL), symptoms, satisfac- 
tion and adherence to medication. These subjective mea- 
sures of outcome help evaluate the burden of disease 
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and treatment from the patients' perspective. In the con- 
ceptual framework developed by Till and colleagues 
adapted in (Figure 1) [3], PRO data from clinical trials 
may directly inform patients and practitioners, or may 
indirectly inform clinical practice through evidence syn- 
thesis into clinical practice guidelines. 

The quality of data, including PROs, from trials may be 
threatened by the trial design, execution, and reporting. 
Like any other outcome being considered, PROs should be 
pre-specified, relevant and appropriate measure(s) [4]. Fail- 
ure to address a number of specific issues at the time of 
protocol writing or in the final trial report, such as the PRO 
hypothesis, timing and mode of assessment and strategies 
to manage missing data, can undermine the clinical validity 
of findings and limit translation of these into clinical prac- 
tice [5]. 

All RCT outcomes, including PROs, should be com- 
pletely and transparently reported. Anything less may di- 
minish the usability of the trial results by clinicians and 
is an inefficient use of resources. High-quality reporting 
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Figure 1 Model for the use of PRO data to inform patient care. Major routes are indicated by solid arrows whilst other possible routes are 
indicated by unfilled arrows. The potential impact of the CONSORT PRO Extension on evidence from clinical trials and links to guideline 
development are indicated. Adapted from Till et al. [3], 



of PROs in clinical trials is required to allow readers to 
assess the validity of the PRO findings and assess any 
potential sources of bias. Evidence suggests, however, 
that reporting of PROs remains sub-optimal across RCTs 
[6,7]. Previous studies estimating the proportion of 
PRO-RCTs that were likely to robustly inform clinical 
decision-makings in prostate cancer, for example, have 
found that only one third of PRO-RCTs published be- 
tween 1980 and 2001 did so [8]. 

In a review of 794 RCTs published up to 2008 across a 
range of clinical settings, only 56% of articles provided a 
rationale for the selected PRO, 50% provided a PRO hy- 
pothesis, 28% provided information about PRO missing 
data and 64% discussed the PRO findings in the context 
of the other trial outcomes [7]. Although the PROs of 
some RCTs were reported in an expanded supplemen- 
tary publication, this only occurred for 15% of trials at 
the time of review, and the detailed PRO publications 
often appeared in methodologically focused journals. In 
cancer RCTs methodological drawbacks have often ham- 
pered a critical appraisal of results [9,10]. 

Additional qualitative evidence suggests that these pat- 
terns of PRO reporting negatively impact on the effective 
use and dissemination of RCT PRO findings to clinical 
practice. A study of 30 academic oncologists examining 
their attitudes and behaviors regarding HRQL data from 
clinical trials indicated that, while these data were typic- 
ally valued by clinicians and felt to benefit their patients, 
problems with the manner in which these data were re- 
ported in the literature limited their use in clinical prac- 
tice [5]. The data were considered to be less accessible 
than outcomes such as survival or the adverse effects of 
treatment. Many clinicians felt that HRQL/PRO data 
were frequently reported in a manner that is difficult to 



interpret and integrate with clinical outcomes, and were 
often reported months or years after the primary trial 
publication. Clinicians also wanted to be assured that 
the appropriate measurement instrument was used, that 
the results were not biased by missing data, that results 
were not "cherry picked" (i.e., chance findings arising 
from multiple testing), and that reported changes in 
PROs were clinically meaningful. 

In this paper, we focus on how the recently developed 
CONSORT (Consolidated Standards of Reporting Trials) 
PRO extension [11] may be implemented to inform indi- 
vidual patient care and clinical guidelines as illustrated 
in (Figure 1). 

The CONSORT PRO extension as a vehicle for change and 
challenges to its implementation 

The CONSORT statement aims to encourage transparent 
and complete reporting of clinical trials and is associated 
with improved reporting of trials [12]. Importandy, the 
CONSORT group has a proven ability to reach consensus 
on reporting guidelines and to promote their successful im- 
plementation. Extensions to the original CONSORT state- 
ment have provided guidance for specific trial designs, 
interventions, and specific outcomes but until recendy did 
not provide specific guidance for PROs [13]. Such guidance 
was clearly required since multiple factors threaten the 
quality of PRO reporting: researchers must design their 
RCTs with appropriate PRO implementation and execu- 
tion, and authors must find a balance between providing 
sufficient detail to adequately report PRO results within the 
context of other study outcomes while still adhering to 
journal requirements for manuscript length and structure. 
Reviewers and editors must find the optimal balance 
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between demanding that certain reporting standards be in- 
cluded in the context of the same journal constraints. 

The development of a CONSORT PRO extension was 
thus undertaken, led by a Task Force of the International 
Society for Quality of Life Research (ISOQOL) and 
guided by the methodological framework proposed by 
the EQUATOR Network [14]. Briefly, a systematic 
search was used to identify candidate reporting stan- 
dards for PROs [15]. An electronic survey of stakeholder 
groups was used to obtain experts' judgments of the im- 
portance of each candidate standard for PRO reporting 
and the survey results were then debated by a face-to- 
face meeting of stakeholders to achieve consensus on 
items that should comprise a CONSORT extension. The 
meeting agenda included: a review of relevant evidence, 
discussion of the rationale for including or excluding 
items in a PRO reporting standards checklist from the 
point of view of the multiple stakeholders at the table, 
anonymous voting on the proposed standards, the devel- 
opment of a draft CONSORT extension, and the devel- 
opment of a knowledge translation and dissemination 
strategy [13,14]. 

The group was ultimately able to reach consensus on 
five "extension" statements to the 2010 CONSORT guid- 
ance that each address a key reporting item considered 
critical for quality reporting from all types of RCTs 
where PROs are a primary or "important" secondary 
outcome as shown in Table 1. Important secondary out- 
comes refer to pre-specified domains that are focus of 



the principal PRO analyses [11]. In addition, critical 
components of the existing 2010 CONSORT guidance 
that were critically relevant to PRO reporting were ex- 
panded by "elaboration" statements. The rationale for 
each of the 5 items comprising the CONSORT PRO ex- 
tension are briefly summarized in Table 1. Readers are 
referred to the CONSORT PRO Extension for further 
information on the rationale for each item, examples of 
reporting and elaboration of the existing CONSORT 
2010 statement in relation to PROs [11]. These guidance 
statements, if implemented by authors and journal re- 
viewers/editors, could potentially improve reporting and 
facilitate interpretation of PRO results for use in clinical 
practice. 

Experience with the CONSORT Statement suggests 
that efforts to implement the CONSORT PRO Extension 
will face a number of challenges. First and foremost is 
obtaining journal endorsement for the CONSORT PRO 
Extension. A 2008 analysis of "Instructions to Authors" 
for 165 high impact factor medical journals found that 
only 38% mentioned the main CONSORT statement in 
their online instructions [16]. Moreover, of the four 
CONSORT extensions that were examined, no single ex- 
tension was mentioned on more than 3% of journal web- 
sites. Evidence suggests that reporting is improved in 
those journals with a policy of active implementation of 
CONSORT [17]. 

Theory and research on dissemination and implemen- 
tation in health care settings suggests ways to promote 



Table 1 Summary of the CONSORT PRO extension items 



CONSORT 2010 statement 



PRO extension 



Brief rationale for extension 



Structured summary of trial design, 
methods, results, and conclusions 

Specific objectives or hypotheses 



Completely defined pre-specified primary 
and secondary outcome measures, 
including how and when they were 
assessed. 

Statistical IVlethods used to compare 
groups for primary and secondary 
outcomes 



Trial limitations addressing sources of 
potential bias, imprecision, and, if relevant 
multiplicity of analyses 

Generalizability (external validity, 
applicability) of trial findings 



The PRO should be identified in the abstract as a 
primary or secondary outcome 

The PRO hypothesis should be stated and relevant 
domains identified, if applicable. 



Evidence of PRO Instrument validity and reliability 
should be provided or cited if available including 
the person completing the PRO and methods of 
data collection (paper telephone electronic other). 

Statistical approaches for dealing with missing data Missing PRO data is a potential source of bias. A 



Explicitly identifying PROs in the RCT abstract will 
facilitate indexing and identification of studies to 
inform clinical care and evidence synthesis. 

PRO measures may be multi-dimensional and may 
assess patient status at several time points during 
a RCT. A pre-specified hypothesis reduces the risk 
of multiple statistical testing and selective 
reporting of PROs based on statistically significant 
results. 

This information will allow readers to assess the 
validity, reliability and appropriateness of the PRO 
being used. 



are explicitly stated. 



PRO-specific limitations and implications for 
generalizability and clinical practice should be 
discussed. 



number of methods for dealing with missing data 
are available with different strengths and 
imitations which should be described to facilitate 
interpretation. 

PRO specific limitations may influence the 
generalizability of results and use in clinical 

practice. 



Authors are referred to the CONSORT PRO Extension paper for further explanation and the full list of items [1 1]. The CONSORT PRO extension should be used in 
conjunction with the CONSORT checklist and relevant extensions as appropriate for the study design. 
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greater adoption of the CONSORT statement and its ex- 
tensions [18]. Success can be enhanced when efforts ex- 
tend beyond typical "push" approaches (i.e., publishing 
and publicizing new information), to include "pull" ap- 
proaches (i.e., creating demand for change among the 
target audience). One possible way to create demand for 
endorsement is to have regular public reporting audit of 
those journals that do and do not mention CONSORT 
or its extensions in their online Instructions to Authors. 

Even when CONSORT is endorsed there is considerable 
variation in how it is implemented at journals. Without 
strong direct language, for example, in a journal's Instruc- 
tions to Authors section, prospective authors will not have 
clear guidance about how they need to comply with journal 
endorsement. Plans to develop online resources which pro- 
vide specific examples taken from publications should fa- 
cilitate adherence to the new CONSORT-PRO checklist. 
Problems may be encountered; however, with consistency 
in the operationalization of "important secondary out- 
comes" since this term is not currently part of the standard 
clinical trial lexicon. In addition, problems may be encoun- 
tered with how authors interpret the requirement that evi- 
dence of PRO instrument reliability or validity be provided 
or cited if available. Several questions arise in this regard. Is 
it sufficient to cite the original article that describes an in- 
strument's psychometric properties? Ideally further infor- 
mation on reliability should be reported in relation to the 
population of study in the clinical trial. Should supporting 
evidence also be provided for an instrument's responsive- 
ness to change? Should additional psychometric informa- 
tion be provided for translations of instruments? In 
addressing these issues, authors will need to stay abreast of 
evolving standards for PROs and their use in clinical re- 
search [2,19]. 



Potential impact of the CONSORT PRO extension on the 
quality of clinical trial reporting 

The need for improvement in the reporting of PROs is one 
of the principal reasons for developing the CONSORT 
PRO Extension. How likely is it, however, that publication 
and implementation of the CONSORT PRO Extension will 
have the desired positive impact? A recent systematic re- 
view and meta-analysis provides some insight. The review 
encompassed published studies evaluating the complete- 
ness of reporting of randomized controlled trials in medical 
journals based on CONSORT Statement criteria [12]. Data 
for the review were drawn from 53 publications encom- 
passing over 16,000 randomized clinical trials. Findings 
showed that 25 of 27 comparisons of the completeness of 
reporting favored CONSORT-endorsing journals over non- 
endorsers, with five of these comparisons yielding statisti- 
cally significant differences. These findings strongly suggest 
that endorsement of the CONSORT PRO Extension will 
over time result in improvements in the reporting of PROs 
in clinical trials. 

A Real-World Example from Oncology: evidence from 
prostate cancer RCTs evaluated against the CONSORT 
PRO extension items 

In an effort to provide current evidence on the level of 
reporting according to the new CONSORT PRO exten- 
sion items, in Table 2 we provide the proportion of re- 
cently conducted PRO-RCTs in prostate cancers that 
have addressed such issues. A systematic literature 
search, identifying all RCTs published with a PRO com- 
ponent (published between 2004 and 2012) in prostate 
cancers found 65 studies [20]. All these RCTs were scru- 
tinized based on a number of methodological criteria, 
including the CONSORT PRO extension items and data 



Table 2 Level of reporting in recently conducted prostate cancer RCTs (65 studies published between 2004 and 2012) 
according to the new CONOSRT PRO extension 



Consort 2010 statement 



PRO extension 



Level of reporting, n (%) 



Structured summary of trial design, methods, 
results, and conclusions 

Specific objectives or hypotheses 



Completely defined pre-specified primary and 
secondary outcome measures, including how 
and when they were assessed. 

Statistical IVlethods used to compare groups for 
primary and secondary outcomes 

Trial limitations addressing sources of potential 
bias, imprecision, and, if relevant multiplicity of 
analyses 

Generalizability (external validity, applicability) of 
trial findings 



The PRO should be identified in the abstract as a primary 
or secondary outcome 

The PRO hypothesis should be stated and relevant 
domains identified, if applicable. 

Evidence of PRO Instrument validity and reliability should 
be provided or cited if available including the person 
completing the PRO and methods of data collection 
(paper telephone electronic other). 

Statistical approaches for dealing with missing data are 
explicitly stated. 



PRO-specific limitations and implications for generalizability 
and clinical practice should be discussed. 



59/65 (919 



24/65 (37%) 

43/65 (66%), Yes, for all PRO 
instruments used in the study. 

15/65 (23%), reported "mode of 
administration" 

12/65 (18%) 

23/65 (35%) PRO-specific limitations 
discussed. 

37/65 (57%) Implications for 
generalizability 



Data extracted from Efficace et al. [20]. 
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are summarized in Table 2. A remarkable finding is that 
only 18% of studies documented statistical methods for 
addressing PRO missing data. Also, only 23% reported 
methods for PRO administration. These very low per- 
centages, underscore that some specific items reported 
in the new CONSORT PRO extension, are in urgent 
need of improvement to raise confidence in the solidity 
of PRO data stemming from these RCTs. 

Implications of the CONSORT PRO extension for RCT 
design 

The CONSORT PRO extension has important implications 
for trial design, although design decisions may go beyond 
simply what should be reported. The problem of PRO 
missing data illustrates this distinction. Difficulties with 
PRO data collection and compliance have historically been 
considered the major barriers to the successful implemen- 
tation of PRO in clinical trials [21]. The CONSORT PRO 
extension states that the amount of missing data and way 
that this has been addressed in the analysis should be re- 
ported. The analysis plan should clearly be pre-specified at 
the design stage, but in addition methods to prevent or 
minimize missing data in the trial should be stated. In 
addition the CONSORT PRO Extension stipulates that evi- 
dence of PRO instrument validity and reliability be included 
as part of the manuscript submission. This criterion has the 
potential to foster greater consideration of a PRO measures 
psychometric properties at the time of protocol design, 
thereby promoting greater use of measures with demon- 
strated reliability and validity. Likewise, consider the CON- 
SORT PRO Extension item stipulating that the PRO 
hypothesis be stated in the manuscript, with relevant do- 
mains identified if applicable. This criterion has the poten- 
tial to foster greater pre-specification of PRO hypotheses at 
the time of protocol development, thereby reducing the 
likelihood of selective reporting of PRO results in manu- 
script. It should be noted, however, that specification of 
outcomes in a protocol may not be sufficient to eliminate 
the problem of selective result reporting. Indeed, an audit 
comparing protocols submitted to a leading medical journal 
and the published reports based on those protocols found 
numerous discrepancies between the pre-specified out- 
comes and the reported outcomes [22]. Additional efforts, 
such as providing manuscript reviewers with access to the 
original protocols, would appear to be necessary to mean- 
ingfially address the issue of selective reporting. 

Within this framework, it is possible to envisage a 
positive impact of endorsement of the CONSORT 
PRO Extension not only in improving to the quality 
of clinical trial reporting but also on PRO trial de- 
sign. The quality of certain aspects of clinical trials 
incorporating PROs are likely to improve as a direct 
result of widespread adoption of the CONSORT 



PRO Extension, while others will require more 
multi-faceted efforts on the part of the research 
community. 

Improved PRO trial design and reporting: implications for 
guideline development 

Individual randomized trials may either by chance or 
as a result of bias not represent the best possible esti- 
mates of an effect of an intervention on PROs. Thus, 
single small RCTs are rarely informative for evidence 
based health care decisions. Evidence-based health 
care decisions require systematic reviews to more 
completely consider the possible influence of bias and 
chance on health care decisions. 

Systematic reviews of intervention effects alone, 
however, are also insufficient for evidence-based 
health care decision-making. In addition to informa- 
tion about effects of interventions, health care deci- 
sions require consideration of issues around the 
clinical state and circumstances including baseline 
risks for a certain outcome, the populations and soci- 
etal values and preferences, estimates of effect and 
expertise to bring all of that together. Practice guide- 
lines are recommendations that are intended to assist 
providers, recipients of health care and other stake- 
holders to make informed decisions and can take on 
the role of adequate support tools [23]. 

PROs can inform evidence-based recommendations 
in a number of ways. PROs address what matters to 
patients and their use in clinical trials can provide 
the information that is critical for decision making in 
practice guidelines. PROs also can provide informa- 
tion about the importance of other outcomes, non 
PROs, by conducting explicit value and preference 
elicitation exercises. Transparent communication of 
both the confidence in an estimate of effect on a 
PRO, which is also called the quality of the evidence 
[24,25], and the estimate itself remains challenging. 
The GRADE Working Group has developed a system 
to assess the confidence and the likelihood of an esti- 
mate of effect on PRO that facilitates communication, 
which may be incorporated formally in the review 
process. The group is now focusing on a structured 
approach to assessing the confidence in estimates of 
values and preferences that influence the development 
of health care recommendations. The system is used 
by over 70 organizations worldwide. Using such sys- 
tems has become a requirement in most standards on 
clinical practice guideline development [26,27]. 

Conclusion 

In conclusion, robust methodology and accurate 
reporting of data are crucial when evaluating PROs in 
clinical trials in order to provide health care providers 
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and policy-makers with a transparent message about 
the impact of a given drug or a novel therapeutic ap- 
proach on the patient's health status. Scientific publi- 
cations stemming from a poor study design or simply 
reporting insufficient information can mislead readers 
when interpreting study outcomes. Such publications 
are wasteful, both in terms of opportunities for pro- 
viding better evidence, research funding, the effort of 
everyone involved from concept to paper - including 
but not confined to investigators, ethics committees, 
patients, trial co-ordinators, statisticians, journal staff, 
reviewers and finally readers. The new CONSORT 
PRO extension aims to raise quality standards and 
help bridge the gap between PRO-trial based out- 
comes and clinical practice by increasing clinician 
confidence in published PRO evidence. 
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